SuperHack/SuperHack CD.bin/CODING/MISC/NUD-1022.ZIP/NUD-1022.TXT

Text File | 1992-12-09 | 28.7 KB | 675 lines

Neuron Digest   Wednesday,  2 Dec 1992
                Volume 10 : Issue 22

Today's Topics:
                        Re: Beginning references
     Re: begining references on NN for Carl Vettore, Univ. of Verona
              Very Fast Simulated Reannealing version 6.20
                Position in cognitive psychology - U Mich
                  Postdocs - computational neuroscience
                         Vision postdoc position
 (1) HKP review in neuroprose and (2) NIPS time series workshop program
               NIPS workshop - REAL biological computation


Send submissions, questions, address maintenance, and requests for old
issues to "[e-mail]". The ftp archives are
available from cattell.psych.upenn.edu (130.91.68.31). Back issues
requested by mail will eventually be sent, but may take a while.

----------------------------------------------------------------------

Subject: Re: Beginning references
From:    [e-mail] (Michael Arbib)
Date:    Sat, 28 Nov 92 15:39:31 -0800

Michael A. Arbib: Brains, Machines and Mathematics, Second Edition,
Springer-Verlag, 1987.  This places connectionism in a historical
setting, introduces some of the main concepts (relatively brief), places
it in the context of theoretical computer science, and includes chapters
on self-reproducing automata and Godel's Incompleteness Theorem (both
proofs and philosophical implications).

It's not the best introduction to connectionism as a standalone subject,
but is unrivalled for readers who want a "situated" view of the subject
along the above lines.

While advertising my books, I should also put in a word for "The
Metaphorical Brain 2: Neural Networks and Beyond", Wiley-Interscience,
1989.  The special strength of this relative to other books on
connectionism is that it provides strong links to both Computational
Neuroscience AND Artificial Intelligence.  It also argues for Schema
Theory: using schemas as a coarse-grain methodology for cooperative
computation to complement the fine-grain methodology of neural networks.
It closes with a view of Sixth Generation Computing based on this
multi-level schema/NN methodology.

Michael Arbib
Center for Neural Engineering
University of Southern California
Los Angeles, CA 90089-2520, USA
Tel: [phone]
Fax: [phone]
[e-mail]


------------------------------

Subject: Re: begining references on NN for Carl Vettore, Univ. of Verona
From:    [e-mail] (Laveen N. Kanal)
Date:    Sun, 29 Nov 92 10:55:43 -0500


The two volumes by Maureen Caudill from M.I.T. Press should be a
good place to start. The complete reference is cited in the following
list of refernces for a course/seminar that I am offering next semester.



.ce4
CONNECTIONISTS MODELING OF INTELLIGENT SYSTEMS
.br
CMSC 727, SPRING 1993
.br
Instructor: Laveen Kanal
.br
Time: Tues & Thurs. 5:00 to 6:15 p.m.
       (If feasible, may be changed to meet once a week for
         a longer period at a time convenient to the attendees)

This is a graduate course/seminar on Neural Networks and other connectionist
dynamical systems and hybrid models currently being proposed for the
modeling of intelligent systems for pattern recognition and problem-solving.
The purpose is to introduce some of the theory and applications of such
systems, discuss their role and potential in developing intelligent
artificial systems, their relationships to other areas of A.I. and
the question of local versus nonlocal representations.
There will be lectures by the instructor, by invited
persons working in these areas, and possibly by some students.
Students registered for the course will
be expected to do some projects using the suggested lab texts or other
simulators. A mid semester report and a final project/paper will be required
for a grade in the course.

Text: Roberto Serra, Gianni Zanarini, COMPLEX SYSTEMS and COGNITIVE PROCESSES
                                      Springer-Verlag, 1990 ISBN 0-387-51393-0

Lab. text: Maureen Caudill, Charles Butler, Understanding Neural Networks:
           Computer Explorations, vol.1 & vo2, The MIT Press
           (These volumes cover most of the major NN paradigms in an easy to
            follow manner and  a PC or MAC software diskette is included).
            Vol 1 ISBN 0-262-53099-6 IBM compatible disk included
                   ISBN 0-262-53102-x Macintosh compatible disk included

             Vol 2: ISBN 0-262-53100-3 IBM compatible disk included
                    ISBN 0-262-53103-8 Macintosh Compatible disk included

Optional lab. text:
            Adam Blum, Neural Networks in C++ , An Object Oriented Approach
            to building Connectionist Systems, John Wiley & Sons, Inc. 1992
            ISBN 0-471-55201-1(book/IBM disk set)
            ISBN 0-471-53847-7(papaerback book only)

Useful
Supplementary text:

   J.Hertz, A. Krogh, R.C. Palmer, An Introduction to the Theory of Neural
                                   Computation, Addison Wesley, 1991
                                    2nd edition to appear shortly.

Supplementary References:

   L.N. Kanal, On Pattern, Categories, and Multiple Reality, TR, UMD, 1993

   L.N. Kanal & S. Raghavan, Hybrid Systems-A Key to Intelligent Pattern
             Recognition, Proc. IJCNN, 1992.

   J. Hendler, Papers on Hybrid Systems

   J.A. Reggia, Papers on Competition-Based Local Processing for Spreading
               Activation Mechanisms in Neural Networks; also Chap.7 in
               Peng and Reggia, Abductive Inference Models for Diagnostic
               Problem-Solving, Springer-Verlag, 1990

   Todd C. Moody, PHILOSOPHY & ARTIFICIAL INTELLIGENCE, Prentice Hall, 1993
                    ISBN  0-13-663816-3

   Satosi Watanabe, PATTERN RECOGNITION: HUMAN & MECHANICAL, John Wiley, 1985
                                      ISBN 0-471-80815-6

   Gerald M. Edelman, BRIGHT AIR,BRILLIANT FIRE...On the Matter of the Mind,
                      Basic Books, 1992 . ISBN 0-465-05245-2

   George Lakoff, WOMEN, FIRE, and DANGEROUS THINGS--What Categories Reveal
                  About the Mind, Univ. of Chicago Press, 1987
                   ISBN 0-226-46803-8

   I.K. Sethi & A.K. Jain (Eds), ARTIFICIAL NEURAL NETWORKS AND STATISTICAL
                  PATTERN RECOGNITION, North-Holland, 1991
                       ISBN 0-444-88741-5 (paperback)

  K. Yasue, M. Jibu, and Karl Pribram, A Theory of Non Local Cortical
     Processing in the Brain, Appendices to K. Pribram, BRAIN & PERCEPTION,
     Lawrence Earlbaum Associates, 1991. ISBN 0-89859-995-4

  P.A. Flach, R.A. Meersman (Eds), FUTURE DIRECTIONS IN A.I., North-Holland,
    1991.  ISBN 0-444-89048-3


  Papers in IEEE Trans. on Neural Networks, the journal Neural Networks,
  Neural Computation, and Proceedings of Conferences on Neural Networks,
  Genetic Algorithms, Complex Systems, and Hybrid Systems.


Hope this helps.

L.K.


------------------------------

Subject: Very Fast Simulated Reannealing version 6.20
From:    Lester Ingber <[e-mail]>
Date:    Mon, 30 Nov 92 07:17:21 -0800

           VERY FAST SIMULATED REANNEALING (VFSR) (C)

             Lester Ingber [e-mail]
                               and
              Bruce Rosen [e-mail]

The good news is that the people who have gotten our beta version of
VFSR to work on their applications are very pleased.  The bad news is
that because of some blunders made in the process of making the code
user-friendly, the code has to be modified to use as a standalone
function call.  This bug is corrected and some other fixes/changes
are made in version v6.20.

This version is now updated in [e-mail].  It will
eventually find its way into the other NETLIB archives.

To access the new version:

Interactive
     local% ftp research.att.com
     Name (research.att.com:your_login_name): netlib
     Password: [type in your_login_name or anything]
     ftp> cd opt
     ftp> binary
     ftp> get vfsr.Z
     ftp> quit
     local% uncompress vfsr.Z
     local% sh vfsr

Electronic Mail Request
     local% mail [e-mail]
     [mail [e-mail]]
     [mail [e-mail]]
     [mail [e-mail]]
     [mail [e-mail]]
     send vfsr from opt
     ^D [or however you send mail]

Lester


  ||  Prof. Lester Ingber               [e-mail]  ||
  ||  P.O. Box 857                                                 ||
  ||  McLean, VA 22101       703-848-1859 = [10ATT]0-700-L-INGBER  ||


------------------------------

Subject: Position in cognitive psychology - U Mich
From:    [e-mail]
Date:    Fri, 27 Nov 92 10:12:04 -0500

Position in Cognitive Psychology
University of Michigan

The University of Michigan Department of Psychology invites applications
for a tenure-track position in the area of Cognition, beginning September
1, 1993.  The appointment will most likely be made at the Assistant
Professor level, but it is possible at any rank.  We seek candidates with
primary interests and technical skills in cognitive psychology.  Our
primary goal is to hire an outstanding cognitive psychologist, and thus
we will look at candidates with any specific research interest.  We have
a preference for candidates interested in higher mental processes or for
candidates with computational modeling skills (including connectionism).
Responsibilities include graduate and undergraduate teaching, as well as
research and research supervision.  Send curriculum vitae, letters of
reference, copies of recent publications, and a statement of research and
teaching interests no later than January 8, 1993 to: Gary Olson, Chair,
Cognitive Processes Search Committee, Department of Psychology,
University of Michigan, 330 Packard Road, Ann Arbor, Michigan 48104.  The
University of Michigan is an Equal Opportunity/Affirmative Action
employer.



------------------------------

Subject: Postdocs - computational neuroscience
From:    Ken Miller <[e-mail]>
Date:    Sun, 29 Nov 92 06:29:29 -0800


                      POSTDOCTORAL POSITIONS
                    COMPUTATIONAL NEUROSCIENCE
             UNIVERSITY OF CALIFORNIA, SAN FRANCISCO

I will soon be beginning a new lab at UCSF, and anticipate several positions
for postdocs beginning in 1993 and 1994 (prospective graduate students are
also encouraged to apply to the UCSF Neuroscience Program).  The lab will
focus on understanding both development and mature processing in the
cerebral cortex.  Theoretical, computational, and experimental approaches
will be taken.  Candidates should have skills relevant to one or more of
those approaches.  The most important criteria are demonstrated scientific
ability and creativity, and a deep interest in grappling with the details of
neurobiology and the brain.

Past work has focused on modeling of development in visual cortex under
Hebbian and similar ``correlation-based" rules of synaptic plasticity.  The
goal has been to understand these rules in a general way that allows
experimental predictions to be made.  Models have been formulated for the
development of ocular dominance and orientation columns.  A few references
are listed below.

Future work of the lab will extend the developmental modeling, and will also
take various approaches to understanding mature cortical function.  These
will include detailed biophysical modeling of visual cortical networks,
many-cell recording from visual cortex, and use of a number of theoretical
methods to guide and interpret this recording.  There will also be
opportunities for theoretical forays in new directions, in particular in
collaborations with the other Neuroscientists at UCSF.  Facilities to
develop new experimental directions that are relevant to the lab's program,
for example slice studies and use of optical methods, will also exist.

I will be part of the Keck Center for Systems Neuroscience at UCSF, which
will be a very interactive environment for Systems Neurobiology.  Other
members will include:
* Alan Basbaum (pain systems);
* Allison Doupe (song learning in songbirds);
* Steve Lisberger (oculomotor system);
* Michael Merzenich (adult cortical plasticity);
* Christof Schreiner (auditory system);
* Michael Stryker (visual system, development and plasticity);
Closely related faculty members include Roger Nicoll (hippocampus, LTP);
Rob Malenka (hippocampus, LTP); Howard Fields (pain systems); and Henry
Ralston (spinal cord and thalamus).

Please send a letter describing your interests and a C.V., and arrange to
have three letters of recommendation sent to

Ken Miller
Division of Biology 216-76
Caltech
Pasadena, CA 91125
[e-mail]

Some References:

Miller, K.D. (1992). ``Models of Activity-Dependent Neural Development."
Seminars in the Neurosciences, 4:61-73.

Miller, K.D. (1992).  ``Development of Orientation Columns Via Competition
Between ON- and OFF-Center Inputs."  NeuroReport 3:73-76.

MacKay, D.J.C. and K.D. Miller (1990).  ``Analysis of Linsker's simulations
of Hebbian rules," Neural Computation 2:169-182.

Miller, K.D. (1990).  ``Correlation-based mechanisms of neural development,"
in Neuroscience and Connectionist Theory, M.A. Gluck and D.E.  Rumelhart,
Eds.  (Lawrence Erlbaum Associates, Hillsdale NJ), pp.  267-353.

Miller, K.D., J.B. Keller and M.P. Stryker (1989).  ``Ocular dominance
column development: analysis and simulation," Science 245:605-615.

Miller, K.D., B. Chapman and M.P. Stryker (1989).  ``Responses of cells in
cat visual cortex depend on NMDA receptors," Proc. Nat. Acad. Sci. USA
86:5183-5187.


------------------------------

Subject: Vision postdoc position
From:    [e-mail] (Ralph Siegel)
Date:    Tue, 01 Dec 92 18:45:54 -0500


                             PLEASE POST

Postdoctoral position available in analysis of structure-from-motion.
(visual psychophysics, electrophysiology, primate studies)
in primates.

Contact: Ralph Siegel
         Center for Molecular and Behavioral Neuroscience
         Rutgers, The State University
         197 University Avenue
         Newark, NJ 07102
         phone: [phone]  x3261
         fax:   [phone]

         email: [e-mail]


Term:    24 months, beginning 2/1/93 or later
Salary:  NIH levels

Please send statement of research interests, curriculum vitae, and
names of three references.


------------------------------

Subject: (1) HKP review in neuroprose and (2) NIPS time series workshop program
From:    Andreas Weigend <[e-mail]>
Date:    Sun, 29 Nov 92 04:30:55 -0700

Two things: (1) a paper in neuroprose and (2) the program for a NIPS workshop.


(1) Book review of Hertz-Krogh-Palmer in neuroprose:

    My 17-page book review (for Artificial Intelligence) is available via ftp,
    ftp archive.cis.ohio-state.edu (anonymous, neuron)
    cd pub/neuroprose
    binary
    get weigend.hkp-review.ps.Z
    (then uncompress and lpr)


(2) The updated program for the time series NIPS workshop at Vail this Friday:

                "Time Series Analysis and Predic____"

     John Moody          Mike Mozer              Andreas Weigend
     [e-mail]   [e-mail]   [e-mail]

 --------------------------------------------------------------------------
|    Several new techniques are now being applied to the problem of        |
|    predicting the future behavior of a temporal sequence and deducing    |
|    properties of the system that produced the time series. Both          |
|    connectionist and non-connectionist techniques will be discussed.     |
|    Issues include:                                                       |
|      - algorithms and architectures,                                     |
|      - model selection,                                                  |
|      - performance measures,                                             |
|      - iterated single-step vs direct multi-step prediction,             |
|      - short term vs long term prediction,                               |
|      - growth of error with prediction time,                             |
|      - presence or absence of deterministic chaos,                       |
|      - number of degrees of freedom of the system,                       |
|      - amount of noise in the data,                                      |
|      - robust prediction and estimation,                                 |
|      - detection and classification of signals in noise, etc.            |
 --------------------------------------------------------------------------

     Intended audience: connectionists active in time series analysis.

     Half the available time has been reserved for discussion and
     informal presentations. Lively audience participation is encouraged.


7:30-9:30  General Overviews (20 minutes each) and Discussion.

                John MOODY: Time Series Modeling: Classical Methods and
                            Nonlinear Generalizations
                Mike MOZER: Neural nets for temporal sequence processing

           Andreas WEIGEND: Ideas from the SFI competition for prediction
                            and analysis

4:30-6:30 Special Topics.   Talks (10-15 minutes each),
                            Discussion and Ad-Hoc Presentations.


The rest of this message contains the abstracts of the talks.

John MOODY <[e-mail]>:
    I present an overview of classical linear timeseries modeling methods,
including AR, MA, ARMA, ARIMA, and state-space representations, and discuss
their strengths and weaknesses. I then describe how nonlinear
generalizations of these models can be constructed.

Mike MOZER <[e-mail]>:
    I present a general taxonomy of neural net architectures for processing
time-varying patterns. This taxonomy subsumes existing architectures in the
literature, and points to several promising architectures that have yet to
be examined.  I also discuss some experiments on predicting future values
of a financial time series (US dollar--Swiss franc exchange rates) from the
Santa Fe competition, and make suggestions for future work on this series.

Andreas WEIGEND <[e-mail]>:
    For prediction, I first present `low-pass embedding', a generalization
of the usual delay line that corresponds to filtered delay coordinates. I
then focus on the estimation of prediction errors, including a generalization
that predicts the evolution of the entire probability density function.
    For analysis, I first present `deterministic vs stochastic plots' (DVM)
and then the information theoretic measure called `redundancy' that allows
characterization of the underlying dynamic system without prediction.

Volker TRESP <[e-mail]>:
    Like many physiological processes, the blood glucose / insulin metabolism
is highly nonlinear and involves multiple time scales and multi-dimensional
interactions. We present a model of the blood glucose / insulin metabolism of
a diabetic patient. The model is a hybrid "compartment" / neural network
model and was trained with data from a diabetic patient using the dynamic
backpropagation algorithm.  We demonstrate how our model can be used both for
prediction of blood glucose levels and control of the patient's therapy.
(Joint work with J. Moody and W.R. Delong)

William FINNOFF:
    In financial and economic applications, data sets are typically small
and noisy. The standard "black box" approach to network learning tends to
overfit the training data and thus generalizes poorly. In this talk, we
will discuss the microeconomic foundations of neural network model
structures used to perform economic forecasting.  Further, we will describe
a variety of extended regularization techniques used to prevent overfitting.

Eric WAN <[e-mail]>:
    A neural network for time series prediction which uses Finite Impulse
Response (FIR) linear filters to provide dynamic interconnectivity is
presented.  The FIR network and associated training algorithm are reviewed.
Examples from the Santa Fe competition in prediction and dynamic modeling
of laboratory data and simulated chaotic time series are used to
demonstrate the potentials of the approach.

Fernando PINEDA <[e-mail]>:
    A fast and elegant numerical algorithm for estimating generalized
dimensions and coarse grained mutual information will be be presented.
The relationship to other more well known algorithms will be discussed.
Examples from the Santa Fe time series analysis competition will be used
to demonstrate how to use the algorithm for choosing delay times for
delay coordinate embeddings.

Fu-Sheng TSUNG: <[e-mail]>:
    When modeling a system that generates a time series, what is known about
the system constrains the architecture of the model. As an example, I will
present a recurrent network model of a lobster neural circuit, discuss what
we learned from the model, where the model failed, and possible improvements
from using a pair of sigmoids as a "near-oscillator" primitive for a neuron.


------------------------------

Subject: NIPS workshop - REAL biological computation
From:    Jim Schwaber <[e-mail]>
Date:    Wed, 25 Nov 92 10:11:01 -0500


- -----------NIPS 92 WORKSHOP----------------------

Real Applications of Real Biological Circuits

                    or

"If back-prop is not enough how will we get more?"

                    or

"Is anybody really getting anywhere with biology?"

- ---------------------------------------------------

When:   Friday, Dec. 4th
====

Intended Audience:   Those interested in detailed biological modeling.
==================    Those interested in nonlinear control.
                    Those interested in neuronal signal processing.
                        Those interested in connecting the above.


Organizers:
===========
Richard Granger           Jim Schwaber
[e-mail]       [e-mail]

Agenda:
=======

Morning Session, 7:30 - 9:30, Brain Control Systems and Chemical
- ---------------                               Process Control

Jim Schwaber            Brainstem reflexes as adaptive controllers
Dupont

Babatunde Ogunnaike     Reverse engineering brain control systems
DuPont

Frank Doyle             Neurons as nonlinear systems for control
Purdue

John Hopfield           Discussant
Caltech

Afternoon Session, 4:30 - 6:30, Real biological modeling, nonlinear
- -----------------             systems and signal processing

Richard Granger         Signal processing in real neural systems:  is
UC Irvine               it applicable?

Gary Green              The single neuron as a nonlinear system - its
Newcastle               Volterra kernels as described by neural networks.


Program:
========

We anticipate that the topic will generate several points of view.
Thus, presenters will restrict themselves to a very, very few slides
intended to make a point for discussion.  Given that there now are
concrete examples of taking biological principles to application, we
expect the discussion will center more on how, and at what level,
rather than whether "reverse engineering the brain" is useful.

Granger (UC Irvine):
- -------
The architectures, performance rules and learning rules of most artificial
neural networks are at odds with the anatomy and physiology of real
biological neural circuitry.  For example, mammalian telencephelon
(forebrain) is characterized by extremely sparse connectivity (~1-5%),
almost entirely lacks dense recurrent connections, and has extensive lateral
local circuit connections; inhibition is delayed-onset and relatively
long-lasting (100s of milliseconds) compared to rapid-onset brief excitation
(10s of milliseconds), and they are not interchangeable.  Excitatory
connections learn, but there is very little evidence for plasticity in
inhibitory connections.  Real synaptic plasticity rules are sensitive to
temporal information, are not Hebbian, and do not contain "supervision"
signals in any form related to those common in ANNs.

These discrepancies between natural and artificial NNs raise the question of
whether such biological details are largely extraneous to the behavioral and
computational utility of neural circuitry, or whether such properties may
yield novel rules that confer useful computational abilities to networks
that use them.  In this workshop we will explicitly analyze the power and
utility of a range of novel algorithms derived from detailed biology, and
illustrate specific industrial applicatons of these algorithms in the fields
of process control and signal processing.

Ogunnaike (DuPont):
- -----------
REVERSE ENGINEERING BRAIN CONTROL SYSTEMS:
EXPLORING THE POTENTIAL FOR APPLICATIONS IN CHEMICAL PROCESS CONTROL.
=====================================================================

The main motivation for our efforts lies in the simple fact that there
are remarkable analogies between the human body and the chemical
process plant.  Furthermore, it is known that while the brain has been
quite successful in performing its task as the central supervisor of
intricate control systems operating under conditions which leave very
little margin for error, the control computer in the chemical process
plant has not been so successful.


We have been concerned with seeking answers to the following question:

``Is it possible to ``reverse engineer'' a biological control system
and use the understanding to develop novel approaches to chemical
process control systems design and analysis?''

Our discussion will provide an overview of the tentative answers we
have to date.  We will first provide a brief summary of the salient
features and main problems of chemical process control; we will then
introduce the biological control system under study (the baroreceptor
vagal reflex); finally we will present an actual industrial process
whose main features indicate that it may benefit from the knowledge
garnered from the neurobiological studies.

Doyle (Purdue):
- ------
We are focusing our research on two levels:
1) Neuron level: investigating novel building blocks for process
modeling applications which are motivated by realistic biological
neurons.
2) Network Level: looking for novel approaches to nonlinear dynamic
scheduling algorithms for process control and modeling (again,
motivated by biological signal processing in the baroreceptor reflex).

Green (Newcastle):
- -------
I would love to tell the NIPS people about Volterra series,
especially as we have now made a connection between neural
networks, Volterra series and the differential geometric
representation of networks. This allows us to say why one, two or
more layers are necessary for a particular analytic problem. We can
also say how to invert nets which are homeomorphic in their
mappings. More importantly for us biologists we can turn the state
equations of membrane currents, using neural networks into
approximate Volterra kernels which I think (!) helps understand the
dynamics. This gives a solution to the differential equations,
albeit an approximate one in practical terms. The equations are
invertible and therefore allow a formal link between current clamp
and voltage clamp at the equation level. The method we have used to
do this is of interest to chem. eng. people because we can use the
same concepts in non-linear control. It appears at first glance
that we can link the everyday use of neural networks to well
established theory through a study of tangent spaces of networks.
We construct a state space model of a plant, calculate the
differential of the rate of change of output with respect to the
input. Calculate the same for a neural network. Compare
coefficients. The solution to the set of simultaneous equations for
the coefficents produces a network which is formally equivalent to
the solution of the original differential equation which defined
the state equations.


We will be making the claim that analytic solutions of non-linear
differential equations is possible using neural networks for
some problems. For all other problems an approximate solution is
possible but the architecture that must be used can be defined.
Last I'll show how this is related to the old techniques using
Volterra series and why the kernels and inverse transforms can be
directly extracted from networks. I think it is a new method of
solving what is a very old problem. All in 20 minutes !


------------------------------

End of Neuron Digest [Volume 10 Issue 22]
*****************************************